Online Analytical Processing (OLAP) is a set of techniques for organizing source data to support rapid responses to complex, multi-dimensional queries of the source data. OLAP techniques are commonly applied to business intelligence and marketing problems in which it is desired to investigate how customers behave according to demographic, psychographic, or other attributes. Other applications of OLAP involve relational reporting and data mining. The output of an OLAP query is typically provided as a table having rows and columns comprising “dimensions” and values of the table cells comprising “measures”. The dimensions may be categorized by searchable terms such as sales, date, clerk, customer name, and the like. The measures comprise data values that correspond to the intersection of particular combinations of the dimensions. For example, a query might request date=May 1st, 2011 and clerk=John Smith. The measures would correspond to all data records where the sales date is May 1st and the clerk name is John Smith.
The set of all possible combinations of the source data dimensions contain the set of responses to all possible queries submitted to the OLAP system. To ensure timely response to OLAP queries, and because of the large number of possible queries that could be submitted, many OLAP systems calculate a predetermined number of the possible OLAP queries. If a query is submitted that matches a pre-calculated response, then the response table can be provided immediately. Typically, queries that are most likely to be received are the subject of pre-calculation. Responses to other queries must be solved on demand, upon receipt.
Efficient processing for responding to queries involves selecting which dimension combinations, or views, are most likely to be requested from users in a query, and pre-calculating responses for rapid query response. Other techniques for OLAP involve combining use of relational tables and specialized storage for improved response. It is known to provide what is called “real-time OLAP”, in which the OLAP database (commonly called the “cube”) is stored in RAM, so that responses to OLAP queries can be calculated on the data stored in RAM and returned in real-time. In this type of OLAP, the size of the OLAP data for searching can be no larger than the size of RAM on the computer machine that hosts the OLAP application.
Several approaches to implementing OLAP exist today and have been in the commercial marketplace for over a decade. As apparent from the discussion above, the existing approaches generally suffer from fundamental limitations on the amount of data that can be searched, the number of dimensions in the database, cardinality of the dimensions, and ability to be updated in real-time as new data become available. These limitations render the existing approaches impractical for new classes of OLAP applications that include live dashboards on customer activity and predictive analytical features.